Search for: All records

Integrating multimodal data such as RGB and LiDAR from multiple views significantly increases computational and communication demands, which can be challenging for resource-constrained autonomous agents while meeting the time-critical deadlines required for various mission-critical applications. To address this challenge, we propose CoOpTex, a collaborative task execution framework designed for cooperative perception in distributed autonomous systems (DAS). CoOpTex contribution is twofold: (a) CoOpTex fuses multiview RGB images to create a panoramic camera view for 2D object detection and utilizes 360° LiDAR for 3D object detection, improving accuracy with a lightweight Graph Neural Network (GNN) that integrates object coordinates from both perspectives, (b) To optimize task execution and meet the deadline, CoOpTex dynamically offloads computationally intensive image stitching tasks to auxiliary devices when available and adjusts frame capture rates for RGB frames based on device mobility and processing capabilities. We implement CoOpTex in real-time on static and mobile heterogeneous autonomous agents, which helps to significantly reduce deadline violations by 100% while improving frame rates for 2D detection by 2.2 times in stationary and 2 times in mobile conditions, demonstrating its effectiveness in enabling real-time cooperative perception. 

Monitoring respiratory rate (RR) is essential for early identification of respiratory and metabolic abnormalities. However, the limitations of contact-based sensors and the lack of reliability in many contactless methods make continuous and accurate monitoring difficult in non-clinical settings. To address these challenges, we introduce RespFormer, an edgeoptimized, motion-guided temporal-frequency multimodal fusion transformer framework for real-time, contactless RR estimation and breathing pattern classification. RespFormer integrates dense optical flow analysis with temporal, statistical, and frequencydomain features derived from video sequences and enhances them through a multi-stage signal processing pipeline. These features are modeled using an ensemble of three time series transformer architectures (ETSformer, Temporal Fusion Transformer, and Informer) to capture distinct aspects of temporal dynamics. A shared attention-based refinement module enhances the feature representations, and final predictions are fused using a stackingbased meta-learner. We validate RespFormer on a multimodal dataset comprising synchronized RGB, NIR, and IR video data, including a custom in-house dataset captured under various conditions. Experimental results demonstrate that RespFormer achieves a mean absolute error (MAE) ≈ 0.98 bpm, improving prediction accuracy by ≈ 11% and reducing memory usage by ≈ 26%, while maintaining real-time inference (≈ 1.22 seconds) on resource-constrained devices. Furthermore, RespFormer accurately classifies breathing patterns (normal, bradypnea, tachypnea, and apnea) with 95% accuracy, underscoring it’s potential for practical application in telemedicine, clinical screening, and low-resource healthcare settings.